According to emollick, Google’s Attention Is All You Need trains in 12 hours on 8 GPUs and hits 28.4 BLEU En-De and 41.8 BLEU En-Fr, reshaping NLP.